2026-09-30 15:22:13,692 [ 84035 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:42, check_args_and_update_paths) 2026-09-30 15:22:13,692 [ 84035 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:86, check_args_and_update_paths) 2026-09-30 15:22:13,692 [ 84035 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:97, check_args_and_update_paths) 2026-09-30 15:22:13,692 [ 84035 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:99, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_5862m6 --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=b74d853ebe76 -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 test_allowed_client_hosts/test.py::test_allowed_host test_allowed_client_hosts/test.py::test_denied_host test_async_load_databases/test.py::test_dependent_tables test_async_load_databases/test.py::test_dict_get_data test_async_load_databases/test.py::test_multiple_tables 'test_attach_backup_from_s3_plain/test.py::test_attach_part[compact0]' 'test_attach_backup_from_s3_plain/test.py::test_attach_part[compact1]' 'test_attach_backup_from_s3_plain/test.py::test_attach_part[wide0]' 'test_attach_backup_from_s3_plain/test.py::test_attach_part[wide1]' test_attach_table_from_s3_plain_readonly/test.py::test_attach_table_from_s3_plain_readonly test_attach_table_normalizer/test.py::test_attach_substr test_attach_table_normalizer/test.py::test_attach_substr_restart test_authentication/test.py::test_authentication_fail test_authentication/test.py::test_authentication_pass test_backup_restore_new/test_shutdown_wait_backup.py::test_shutdown_wait_backup 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False]' 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True]' 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True]' test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes test_backup_restore_on_cluster/test.py::test_empty_replicated_table test_backup_restore_on_cluster/test.py::test_file_deduplication test_backup_restore_on_cluster/test.py::test_get_error_from_other_host test_backup_restore_on_cluster/test.py::test_keeper_value_max_size test_backup_restore_on_cluster/test.py::test_mutation test_backup_restore_on_cluster/test.py::test_projection test_backup_restore_on_cluster/test.py::test_replicated_database test_backup_restore_on_cluster/test.py::test_replicated_database_async test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts 'test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database]' 'test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid]' test_backup_restore_on_cluster/test.py::test_replicated_table test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath test_backup_restore_on_cluster/test.py::test_required_privileges 'test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False]' 'test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True]' test_backup_restore_on_cluster/test.py::test_system_functions test_backup_restore_on_cluster/test.py::test_system_users test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty test_backup_restore_on_cluster/test.py::test_tables_dependency test_backward_compatibility/test_cte_distributed.py::test_cte_distributed test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible test_backward_compatibility/test_normalized_count_comparison.py::test_select_aggregate_alias_column test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts test_block_structure_mismatch/test.py::test test_broken_projections/test.py::test_broken_ignored test_broken_projections/test.py::test_broken_ignored_replicated test_broken_projections/test.py::test_broken_on_start test_broken_projections/test.py::test_broken_projections_in_backups_2 test_broken_projections/test.py::test_broken_projections_in_backups_3 test_broken_projections/test.py::test_check_part_thread test_broken_projections/test.py::test_disappeared_projection_on_start test_broken_projections/test.py::test_materialize_broken_projection test_broken_projections/test.py::test_mutation_with_broken_projection test_buffer_profile/test.py::test_buffer_profile test_buffer_profile/test.py::test_default_profile test_cgroup_limit/test.py::test_cgroup_cpu_limit 'test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node]' 'test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries]' test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded]' test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True]' test_cleanup_after_start/test.py::test_old_dirs_cleanup test_concurrent_queries_restriction_by_query_kind/test.py::test_insert test_concurrent_queries_restriction_by_query_kind/test.py::test_select test_config_hide_in_preprocessed/test.py::test_hide_in_preprocessed test_config_xml_main/test.py::test_xml_main_conf test_covered_by_broken_exists/test.py::test_make_clone_covered_by_broken_detached_dir_exists -vvv" altinityinfra/integration-tests-runner:cf95077f4609 '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: repeat-0.9.4, order-1.0.1, random-0.2, timeout-2.2.0, reportlog-0.4.0, xdist-3.5.0 timeout: 900.0s timeout method: signal timeout func_only: False created: 10/10 workers 10 workers [100 items] scheduling tests via LoadFileScheduling test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False] test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node] test_allowed_client_hosts/test.py::test_allowed_host test_async_load_databases/test.py::test_dependent_tables test_authentication/test.py::test_authentication_fail test_buffer_profile/test.py::test_buffer_profile test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible test_attach_table_normalizer/test.py::test_attach_substr test_broken_projections/test.py::test_broken_ignored test_attach_backup_from_s3_plain/test.py::test_attach_part[compact0] [gw7] [ 1%] PASSED test_authentication/test.py::test_authentication_fail test_authentication/test.py::test_authentication_pass [gw6] [ 2%] PASSED test_attach_table_normalizer/test.py::test_attach_substr test_attach_table_normalizer/test.py::test_attach_substr_restart [gw7] [ 3%] PASSED test_authentication/test.py::test_authentication_pass [gw9] [ 4%] PASSED test_buffer_profile/test.py::test_buffer_profile test_buffer_profile/test.py::test_default_profile [gw9] [ 5%] PASSED test_buffer_profile/test.py::test_default_profile [gw6] [ 6%] PASSED test_attach_table_normalizer/test.py::test_attach_substr_restart test_backup_restore_new/test_shutdown_wait_backup.py::test_shutdown_wait_backup test_attach_table_from_s3_plain_readonly/test.py::test_attach_table_from_s3_plain_readonly test_backward_compatibility/test_normalized_count_comparison.py::test_select_aggregate_alias_column [gw4] [ 7%] PASSED test_allowed_client_hosts/test.py::test_allowed_host test_allowed_client_hosts/test.py::test_denied_host [gw4] [ 8%] PASSED test_allowed_client_hosts/test.py::test_denied_host test_concurrent_queries_restriction_by_query_kind/test.py::test_insert [gw3] [ 9%] PASSED test_broken_projections/test.py::test_broken_ignored test_broken_projections/test.py::test_broken_ignored_replicated [gw5] [ 10%] PASSED test_async_load_databases/test.py::test_dependent_tables test_async_load_databases/test.py::test_dict_get_data [gw3] [ 11%] PASSED test_broken_projections/test.py::test_broken_ignored_replicated test_broken_projections/test.py::test_broken_on_start [gw3] [ 12%] PASSED test_broken_projections/test.py::test_broken_on_start test_broken_projections/test.py::test_broken_projections_in_backups_2 [gw1] [ 13%] PASSED test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False] test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True] [gw1] [ 14%] PASSED test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True] [gw5] [ 15%] PASSED test_async_load_databases/test.py::test_dict_get_data test_async_load_databases/test.py::test_multiple_tables test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True] [gw1] [ 16%] PASSED test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True] [gw3] [ 17%] PASSED test_broken_projections/test.py::test_broken_projections_in_backups_2 test_broken_projections/test.py::test_broken_projections_in_backups_3 test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica [gw4] [ 18%] PASSED test_concurrent_queries_restriction_by_query_kind/test.py::test_insert test_concurrent_queries_restriction_by_query_kind/test.py::test_select [gw1] [ 19%] PASSED test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes [gw3] [ 20%] PASSED test_broken_projections/test.py::test_broken_projections_in_backups_3 test_broken_projections/test.py::test_check_part_thread [gw1] [ 21%] PASSED test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes test_backup_restore_on_cluster/test.py::test_empty_replicated_table [gw5] [ 22%] PASSED test_async_load_databases/test.py::test_multiple_tables [gw3] [ 23%] PASSED test_broken_projections/test.py::test_check_part_thread test_broken_projections/test.py::test_disappeared_projection_on_start [gw1] [ 24%] PASSED test_backup_restore_on_cluster/test.py::test_empty_replicated_table test_backup_restore_on_cluster/test.py::test_file_deduplication [gw1] [ 25%] PASSED test_backup_restore_on_cluster/test.py::test_file_deduplication [gw7] [ 26%] PASSED test_backup_restore_new/test_shutdown_wait_backup.py::test_shutdown_wait_backup test_backup_restore_on_cluster/test.py::test_get_error_from_other_host [gw1] [ 27%] PASSED test_backup_restore_on_cluster/test.py::test_get_error_from_other_host test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts test_backup_restore_on_cluster/test.py::test_keeper_value_max_size [gw8] [ 28%] PASSED test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible [gw3] [ 29%] PASSED test_broken_projections/test.py::test_disappeared_projection_on_start test_broken_projections/test.py::test_materialize_broken_projection [gw8] [ 30%] PASSED test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible [gw4] [ 31%] PASSED test_concurrent_queries_restriction_by_query_kind/test.py::test_select [gw9] [ 32%] PASSED test_backward_compatibility/test_normalized_count_comparison.py::test_select_aggregate_alias_column [gw1] [ 33%] PASSED test_backup_restore_on_cluster/test.py::test_keeper_value_max_size test_backup_restore_on_cluster/test.py::test_mutation [gw3] [ 34%] PASSED test_broken_projections/test.py::test_materialize_broken_projection test_broken_projections/test.py::test_mutation_with_broken_projection test_cgroup_limit/test.py::test_cgroup_cpu_limit test_covered_by_broken_exists/test.py::test_make_clone_covered_by_broken_detached_dir_exists test_backward_compatibility/test_cte_distributed.py::test_cte_distributed [gw1] [ 35%] PASSED test_backup_restore_on_cluster/test.py::test_mutation test_backup_restore_on_cluster/test.py::test_projection [gw1] [ 36%] PASSED test_backup_restore_on_cluster/test.py::test_projection test_backup_restore_on_cluster/test.py::test_replicated_database [gw1] [ 37%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_database test_backup_restore_on_cluster/test.py::test_replicated_database_async [gw3] [ 38%] PASSED test_broken_projections/test.py::test_mutation_with_broken_projection [gw1] [ 39%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_async test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts [gw1] [ 40%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database] [gw1] [ 41%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database] test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid] [gw1] [ 42%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid] test_backup_restore_on_cluster/test.py::test_replicated_table [gw1] [ 43%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table [gw8] [ 44%] PASSED test_backward_compatibility/test_cte_distributed.py::test_cte_distributed test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster [gw1] [ 45%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster [gw1] [ 46%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def [gw4] [ 47%] PASSED test_covered_by_broken_exists/test.py::test_make_clone_covered_by_broken_detached_dir_exists [gw1] [ 48%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert [gw1] [ 49%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge [gw1] [ 50%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge [gw7] [ 51%] PASSED test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath test_cleanup_after_start/test.py::test_old_dirs_cleanup [gw1] [ 52%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath test_backup_restore_on_cluster/test.py::test_required_privileges [gw9] [ 53%] PASSED test_cgroup_limit/test.py::test_cgroup_cpu_limit test_config_xml_main/test.py::test_xml_main_conf [gw1] [ 54%] PASSED test_backup_restore_on_cluster/test.py::test_required_privileges test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False] [gw1] [ 55%] PASSED test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False] test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True] [gw1] [ 56%] PASSED test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True] test_backup_restore_on_cluster/test.py::test_system_functions [gw1] [ 57%] PASSED test_backup_restore_on_cluster/test.py::test_system_functions test_backup_restore_on_cluster/test.py::test_system_users [gw1] [ 58%] PASSED test_backup_restore_on_cluster/test.py::test_system_users test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def [gw1] [ 59%] PASSED test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty [gw9] [ 60%] PASSED test_config_xml_main/test.py::test_xml_main_conf [gw1] [ 61%] PASSED test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty test_backup_restore_on_cluster/test.py::test_tables_dependency [gw7] [ 62%] PASSED test_cleanup_after_start/test.py::test_old_dirs_cleanup [gw1] [ 63%] PASSED test_backup_restore_on_cluster/test.py::test_tables_dependency [gw0] [ 64%] ERROR test_attach_backup_from_s3_plain/test.py::test_attach_part[compact0] test_attach_backup_from_s3_plain/test.py::test_attach_part[compact1] [gw0] [ 65%] ERROR test_attach_backup_from_s3_plain/test.py::test_attach_part[compact1] test_attach_backup_from_s3_plain/test.py::test_attach_part[wide0] [gw0] [ 66%] ERROR test_attach_backup_from_s3_plain/test.py::test_attach_part[wide0] test_attach_backup_from_s3_plain/test.py::test_attach_part[wide1] [gw0] [ 67%] ERROR test_attach_backup_from_s3_plain/test.py::test_attach_part[wide1] [gw2] [ 68%] ERROR test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node] test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries] [gw2] [ 69%] ERROR test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries] test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk [gw2] [ 70%] ERROR test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries [gw2] [ 71%] ERROR test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works [gw2] [ 72%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br] [gw2] [ 73%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2] [gw2] [ 74%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate] [gw2] [ 75%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip] [gw2] [ 76%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4] [gw2] [ 77%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none] [gw2] [ 78%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz] [gw2] [ 79%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd] [gw2] [ 80%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br] [gw2] [ 81%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2] [gw2] [ 82%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate] [gw2] [ 83%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip] [gw2] [ 84%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4] [gw2] [ 85%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none] [gw2] [ 86%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz] [gw2] [ 87%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd] [gw2] [ 88%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd] test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused] [gw2] [ 89%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused] test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded] [gw2] [ 90%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded] test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down] [gw2] [ 91%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down] test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded] [gw2] [ 92%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded] test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried [gw2] [ 93%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False] [gw2] [ 94%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False] test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True] [gw2] [ 95%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True] test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False] [gw2] [ 96%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False] test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True] [gw2] [ 97%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True] [gw6] [ 98%] ERROR test_attach_table_from_s3_plain_readonly/test.py::test_attach_table_from_s3_plain_readonly test_block_structure_mismatch/test.py::test [gw6] [ 99%] PASSED test_block_structure_mismatch/test.py::test test_config_hide_in_preprocessed/test.py::test_hide_in_preprocessed [gw6] [100%] PASSED test_config_hide_in_preprocessed/test.py::test_hide_in_preprocessed ==================================== ERRORS ==================================== _________________ ERROR at setup of test_attach_part[compact0] _________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module", autouse=True) def start_cluster(): try: > cluster.start() test_attach_backup_from_s3_plain/test.py:18: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node ... E Pulling node ... pulling from altinityinfra/integr... E Pulling resolver ... pulling from altinityinfra/python... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling minio1 ... error E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ---------------------------- Captured stderr setup ----------------------------- Traceback (most recent call last): File "/ClickHouse/tests/integration/helpers/cluster.py", line 2831, in start retry(log_function=logging_pulling_images)(run_and_check)( File "/ClickHouse/tests/integration/helpers/retry_decorator.py", line 28, in wrapper raise e File "/ClickHouse/tests/integration/helpers/retry_decorator.py", line 19, in wrapper func(*args, **kwargs) File "/ClickHouse/tests/integration/helpers/cluster.py", line 129, in run_and_check raise Exception( Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... Pulling proxy2 ... Pulling resolver ... Pulling minio1 ... Pulling node ... Pulling node ... pulling from altinityinfra/integr... Pulling resolver ... pulling from altinityinfra/python... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node ... done Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy2 ... done Pulling proxy1 ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied ------------------------------ Captured log setup ------------------------------ 2026-09-30 15:22:18 [ 511 ] DEBUG : Command:['docker ps | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:22:18 [ 511 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-09-30 15:22:18 [ 511 ] DEBUG : No running containers (conftest.py:95, cleanup_environment) 2026-09-30 15:22:18 [ 511 ] DEBUG : Pruning Docker networks (conftest.py:97, cleanup_environment) 2026-09-30 15:22:18 [ 511 ] DEBUG : Command:['docker network prune --force'] (cluster.py:113, run_and_check) 2026-09-30 15:22:18 [ 511 ] DEBUG : Command:["sysctl net.ipv4.ip_local_port_range='55000 65535'"] (cluster.py:113, run_and_check) 2026-09-30 15:22:18 [ 511 ] DEBUG : Stdout:net.ipv4.ip_local_port_range = 55000 65535 (cluster.py:121, run_and_check) 2026-09-30 15:22:18 [ 511 ] INFO : Running tests in /ClickHouse/tests/integration/test_attach_backup_from_s3_plain/test.py (cluster.py:2790, start) 2026-09-30 15:22:18 [ 511 ] DEBUG : Cluster start called. is_up=False (cluster.py:2797, start) 2026-09-30 15:22:18 [ 511 ] DEBUG : Docker networks for project roottestattachbackupfroms3plain_gw0 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:22:18 [ 511 ] DEBUG : Docker containers for project roottestattachbackupfroms3plain_gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:22:18 [ 511 ] DEBUG : Docker volumes for project roottestattachbackupfroms3plain_gw0 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:22:18 [ 511 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-09-30 15:22:18 [ 511 ] DEBUG : Docker networks for project roottestattachbackupfroms3plain_gw0 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:22:18 [ 511 ] DEBUG : Docker containers for project roottestattachbackupfroms3plain_gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:22:18 [ 511 ] DEBUG : Docker volumes for project roottestattachbackupfroms3plain_gw0 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:22:18 [ 511 ] DEBUG : Command:docker container list --all --filter name='^/roottestattachbackupfroms3plain_gw0_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-09-30 15:22:18 [ 511 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-09-30 15:22:18 [ 511 ] DEBUG : No running containers for project: roottestattachbackupfroms3plain_gw0 (cluster.py:904, cleanup) 2026-09-30 15:22:18 [ 511 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-09-30 15:22:18 [ 511 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-09-30 15:22:18 [ 511 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:22:18 [ 511 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-09-30 15:22:18 [ 511 ] DEBUG : Setup directory for instance: node (cluster.py:2810, start) 2026-09-30 15:22:18 [ 511 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-09-30 15:22:18 [ 511 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-09-30 15:22:18 [ 511 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-09-30 15:22:18 [ 511 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-09-30 15:22:18 [ 511 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/configs/disk_s3.xml'] to /ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/configs/config.d (cluster.py:4636, create_dir) 2026-09-30 15:22:18 [ 511 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/database (cluster.py:4653, create_dir) 2026-09-30 15:22:18 [ 511 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/logs (cluster.py:4664, create_dir) 2026-09-30 15:22:18 [ 511 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log"] (cluster.py:4750, create_dir) 2026-09-30 15:22:18 [ 511 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'MINIO_CERTS_DIR': '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/minio/certs', 'MINIO_DATA_DIR': '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/minio/data', 'MINIO_PORT': '9001', 'SSL_CERT_FILE': '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/minio/certs/public.crt'} stored in /ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env (cluster.py:86, _create_env_file) 2026-09-30 15:22:18 [ 511 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-09-30 15:22:18 [ 511 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-09-30 15:22:18 [ 511 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-09-30 15:22:18 [ 511 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-09-30 15:22:18 [ 511 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 825 (connectionpool.py:547, _make_request) 2026-09-30 15:22:18 [ 511 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling node ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:23:21 [ 511 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:23:21 [ 511 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... Pulling proxy1 ... Pulling resolver ... Pulling minio1 ... Pulling node ... Pulling node ... pulling from altinityinfra/integr... Pulling resolver ... pulling from altinityinfra/python... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling node ... done Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling resolver ... done Pulling proxy2 ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:23:23 [ 511 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling node ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:24:14 [ 511 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:24:14 [ 511 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... Pulling proxy1 ... Pulling resolver ... Pulling minio1 ... Pulling node ... Pulling resolver ... pulling from altinityinfra/python... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling node ... pulling from altinityinfra/integr... Pulling resolver ... done Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling proxy2 ... done Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:24:16 [ 511 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling node ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:25:17 [ 511 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:25:17 [ 511 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... Pulling proxy2 ... Pulling minio1 ... Pulling node ... Pulling resolver ... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling node ... pulling from altinityinfra/integr... Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling resolver ... pulling from altinityinfra/python... Pulling proxy2 ... done Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node ... done Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:25:20 [ 511 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling node ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:26:12 [ 511 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:26:12 [ 511 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... Pulling proxy2 ... Pulling minio1 ... Pulling node ... Pulling resolver ... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling resolver ... pulling from altinityinfra/python... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy2 ... done Pulling node ... pulling from altinityinfra/integr... Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:26:17 [ 511 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling node ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:27:15 [ 511 ] DEBUG : Failed to start cluster: (cluster.py:3178, start) 2026-09-30 15:27:15 [ 511 ] DEBUG : Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... Pulling proxy2 ... Pulling resolver ... Pulling minio1 ... Pulling node ... Pulling node ... pulling from altinityinfra/integr... Pulling resolver ... pulling from altinityinfra/python... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node ... done Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy2 ... done Pulling proxy1 ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:3179, start) 2026-09-30 15:27:15 [ 511 ] DEBUG : None (cluster.py:3180, start) 2026-09-30 15:27:15 [ 511 ] WARNING : docker-compose up was not called. Trying to export docker.log for running containers (cluster.py:3246, shutdown) 2026-09-30 15:27:15 [ 511 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-09-30 15:27:15 [ 511 ] DEBUG : Docker networks for project roottestattachbackupfroms3plain_gw0 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:27:15 [ 511 ] DEBUG : Docker containers for project roottestattachbackupfroms3plain_gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:27:15 [ 511 ] DEBUG : Docker volumes for project roottestattachbackupfroms3plain_gw0 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:27:15 [ 511 ] DEBUG : Command:docker container list --all --filter name='^/roottestattachbackupfroms3plain_gw0_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-09-30 15:27:16 [ 511 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-09-30 15:27:16 [ 511 ] DEBUG : No running containers for project: roottestattachbackupfroms3plain_gw0 (cluster.py:904, cleanup) 2026-09-30 15:27:16 [ 511 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-09-30 15:27:16 [ 511 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-09-30 15:27:16 [ 511 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:27:16 [ 511 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-09-30 15:27:16 [ 511 ] WARNING : docker-compose up was not called. Trying to export docker.log for running containers (cluster.py:3246, shutdown) 2026-09-30 15:27:16 [ 511 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-09-30 15:27:16 [ 511 ] DEBUG : Docker networks for project roottestattachbackupfroms3plain_gw0 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:27:16 [ 511 ] DEBUG : Docker containers for project roottestattachbackupfroms3plain_gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:27:16 [ 511 ] DEBUG : Docker volumes for project roottestattachbackupfroms3plain_gw0 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:27:16 [ 511 ] DEBUG : Command:docker container list --all --filter name='^/roottestattachbackupfroms3plain_gw0_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-09-30 15:27:16 [ 511 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-09-30 15:27:16 [ 511 ] DEBUG : No running containers for project: roottestattachbackupfroms3plain_gw0 (cluster.py:904, cleanup) 2026-09-30 15:27:16 [ 511 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-09-30 15:27:16 [ 511 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-09-30 15:27:16 [ 511 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:27:16 [ 511 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) _________________ ERROR at setup of test_attach_part[compact1] _________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module", autouse=True) def start_cluster(): try: > cluster.start() test_attach_backup_from_s3_plain/test.py:18: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node ... E Pulling node ... pulling from altinityinfra/integr... E Pulling resolver ... pulling from altinityinfra/python... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling minio1 ... error E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception __________________ ERROR at setup of test_attach_part[wide0] ___________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module", autouse=True) def start_cluster(): try: > cluster.start() test_attach_backup_from_s3_plain/test.py:18: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node ... E Pulling node ... pulling from altinityinfra/integr... E Pulling resolver ... pulling from altinityinfra/python... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling minio1 ... error E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception __________________ ERROR at setup of test_attach_part[wide1] ___________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module", autouse=True) def start_cluster(): try: > cluster.start() test_attach_backup_from_s3_plain/test.py:18: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node ... E Pulling node ... pulling from altinityinfra/integr... E Pulling resolver ... pulling from altinityinfra/python... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling minio1 ... error E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ________________ ERROR at setup of test_adaptive_timeouts[node] ________________ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ---------------------------- Captured stderr setup ----------------------------- Traceback (most recent call last): File "/ClickHouse/tests/integration/helpers/cluster.py", line 2831, in start retry(log_function=logging_pulling_images)(run_and_check)( File "/ClickHouse/tests/integration/helpers/retry_decorator.py", line 28, in wrapper raise e File "/ClickHouse/tests/integration/helpers/retry_decorator.py", line 19, in wrapper func(*args, **kwargs) File "/ClickHouse/tests/integration/helpers/cluster.py", line 129, in run_and_check raise Exception( Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... Pulling proxy1 ... Pulling minio1 ... Pulling node ... Pulling node_with_inf_s3_retries ... Pulling resolver ... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling node ... pulling from altinityinfra/integr... Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy2 ... done Pulling proxy1 ... done Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node ... done Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... Pulling node_with_inf_s3_retries ... status: image is up to date for a... Pulling node_with_inf_s3_retries ... done Pulling minio1 ... error Pulling resolver ... pulling from altinityinfra/python... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied ------------------------------ Captured log setup ------------------------------ 2026-09-30 15:22:18 [ 517 ] DEBUG : Command:['docker ps | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:22:18 [ 517 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-09-30 15:22:18 [ 517 ] DEBUG : No running containers (conftest.py:95, cleanup_environment) 2026-09-30 15:22:18 [ 517 ] DEBUG : Pruning Docker networks (conftest.py:97, cleanup_environment) 2026-09-30 15:22:18 [ 517 ] DEBUG : Command:['docker network prune --force'] (cluster.py:113, run_and_check) 2026-09-30 15:22:18 [ 517 ] DEBUG : Command:["sysctl net.ipv4.ip_local_port_range='55000 65535'"] (cluster.py:113, run_and_check) 2026-09-30 15:22:18 [ 517 ] DEBUG : Stdout:net.ipv4.ip_local_port_range = 55000 65535 (cluster.py:121, run_and_check) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV DOCKER_KERBEROS_KDC_TAG 9391ecdee8d7 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV CLICKHOUSE_TESTS_SERVER_BIN_PATH /clickhouse (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV MSAN_OPTIONS abort_on_error=1 poison_in_dtor=1 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV JAVA_TOOL_OPTIONS -Djdk.attach.allowAttachSelf=true (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV TSAN_OPTIONS halt_on_error=1 abort_on_error=1 history_size=7 memory_limit_mb=46080 second_deadlock_stack=1 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV HOSTNAME 496b253c8ca1 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV SHLVL 0 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV HOME /root (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV OLDPWD / (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV DOCKER_HELPER_TAG 2cffe1eae894 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV PYTHONUNBUFFERED 1 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV DOCKER_PYTHON_BOTTLE_TAG caad4729259e (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV UBSAN_OPTIONS print_stacktrace=1 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV PYTEST_ADDOPTS --dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 test_allowed_client_hosts/test.py::test_allowed_host test_allowed_client_hosts/test.py::test_denied_host test_async_load_databases/test.py::test_dependent_tables test_async_load_databases/test.py::test_dict_get_data test_async_load_databases/test.py::test_multiple_tables 'test_attach_backup_from_s3_plain/test.py::test_attach_part[compact0]' 'test_attach_backup_from_s3_plain/test.py::test_attach_part[compact1]' 'test_attach_backup_from_s3_plain/test.py::test_attach_part[wide0]' 'test_attach_backup_from_s3_plain/test.py::test_attach_part[wide1]' test_attach_table_from_s3_plain_readonly/test.py::test_attach_table_from_s3_plain_readonly test_attach_table_normalizer/test.py::test_attach_substr test_attach_table_normalizer/test.py::test_attach_substr_restart test_authentication/test.py::test_authentication_fail test_authentication/test.py::test_authentication_pass test_backup_restore_new/test_shutdown_wait_backup.py::test_shutdown_wait_backup 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False]' 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True]' 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True]' test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes test_backup_restore_on_cluster/test.py::test_empty_replicated_table test_backup_restore_on_cluster/test.py::test_file_deduplication test_backup_restore_on_cluster/test.py::test_get_error_from_other_host test_backup_restore_on_cluster/test.py::test_keeper_value_max_size test_backup_restore_on_cluster/test.py::test_mutation test_backup_restore_on_cluster/test.py::test_projection test_backup_restore_on_cluster/test.py::test_replicated_database test_backup_restore_on_cluster/test.py::test_replicated_database_async test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts 'test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database]' 'test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid]' test_backup_restore_on_cluster/test.py::test_replicated_table test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath test_backup_restore_on_cluster/test.py::test_required_privileges 'test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False]' 'test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True]' test_backup_restore_on_cluster/test.py::test_system_functions test_backup_restore_on_cluster/test.py::test_system_users test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty test_backup_restore_on_cluster/test.py::test_tables_dependency test_backward_compatibility/test_cte_distributed.py::test_cte_distributed test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible test_backward_compatibility/test_normalized_count_comparison.py::test_select_aggregate_alias_column test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts test_block_structure_mismatch/test.py::test test_broken_projections/test.py::test_broken_ignored test_broken_projections/test.py::test_broken_ignored_replicated test_broken_projections/test.py::test_broken_on_start test_broken_projections/test.py::test_broken_projections_in_backups_2 test_broken_projections/test.py::test_broken_projections_in_backups_3 test_broken_projections/test.py::test_check_part_thread test_broken_projections/test.py::test_disappeared_projection_on_start test_broken_projections/test.py::test_materialize_broken_projection test_broken_projections/test.py::test_mutation_with_broken_projection test_buffer_profile/test.py::test_buffer_profile test_buffer_profile/test.py::test_default_profile test_cgroup_limit/test.py::test_cgroup_cpu_limit 'test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node]' 'test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries]' test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded]' test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True]' test_cleanup_after_start/test.py::test_old_dirs_cleanup test_concurrent_queries_restriction_by_query_kind/test.py::test_insert test_concurrent_queries_restriction_by_query_kind/test.py::test_select test_config_hide_in_preprocessed/test.py::test_hide_in_preprocessed test_config_xml_main/test.py::test_xml_main_conf test_covered_by_broken_exists/test.py::test_make_clone_covered_by_broken_detached_dir_exists -vvv (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV CLICKHOUSE_LIBRARY_BRIDGE_BINARY_PATH /clickhouse-library-bridge (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV COMPOSE_HTTP_TIMEOUT 600 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV DOCKER_MYSQL_PHP_CLIENT_TAG 88be89c1e3b6 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV DOCKER_DOTNET_CLIENT_TAG 11de0b29a15d (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV CLICKHOUSE_TESTS_CLIENT_BIN_PATH /clickhouse (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV DOCKER_MYSQL_JS_CLIENT_TAG 41ba7c2ec2a1 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV PATH /spark-3.3.2-bin-hadoop3/bin:/opt/gdb/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV DOCKER_KERBERIZED_HADOOP_TAG latest (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV DOCKER_CHANNEL stable (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV DOCKER_CLIENT_TIMEOUT 300 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV DOCKER_POSTGRESQL_JAVA_CLIENT_TAG a4eff5c7f4d6 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV DOCKER_NGINX_DAV_TAG b55ac9cd7519 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV DOCKER_MYSQL_GOLANG_CLIENT_TAG 9bec2a638e6e (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV PWD /ClickHouse/tests/integration (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV DOCKER_MYSQL_JAVA_CLIENT_TAG 766bff31cfe4 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV CLICKHOUSE_ODBC_BRIDGE_BINARY_PATH /clickhouse-odbc-bridge (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV CLICKHOUSE_TESTS_BASE_CONFIG_DIR /clickhouse-config (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV TZ Etc/UTC (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV JAVA_PATH /usr/lib/jvm/java-11-openjdk-amd64/bin/java (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV DOCKER_BASE_TAG b74d853ebe76 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV SPARK_HOME /spark-3.3.2-bin-hadoop3 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV LC_CTYPE C.UTF-8 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV INTEGRATION_TESTS_RUN_ID 0 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV WORKER_FREE_PORTS 30100 30101 30102 30103 30104 30105 30106 30107 30108 30109 30110 30111 30112 30113 30114 30115 30116 30117 30118 30119 30120 30121 30122 30123 30124 30125 30126 30127 30128 30129 30130 30131 30132 30133 30134 30135 30136 30137 30138 30139 30140 30141 30142 30143 30144 30145 30146 30147 30148 30149 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV PYTEST_XDIST_TESTRUNUID e6c700aa1a8746fda438af8eb938d565 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV PYTEST_XDIST_WORKER gw2 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV PYTEST_XDIST_WORKER_COUNT 10 (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : ENV PYTEST_CURRENT_TEST test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node] (setup) (cluster.py:446, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : CLUSTER INIT base_config_dir:/clickhouse-config (cluster.py:766, __init__) 2026-09-30 15:22:18 [ 517 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-09-30 15:22:18 [ 517 ] DEBUG : Cluster name: project_name:roottestcheckings3blobsparanoid_gw2. Added instance name:node tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-09-30 15:22:18 [ 517 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-09-30 15:22:18 [ 517 ] DEBUG : Cluster name: project_name:roottestcheckings3blobsparanoid_gw2. Added instance name:node_with_inf_s3_retries tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-09-30 15:22:18 [ 517 ] INFO : Starting cluster... (test.py:41, cluster) 2026-09-30 15:22:18 [ 517 ] INFO : Running tests in /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/test.py (cluster.py:2790, start) 2026-09-30 15:22:18 [ 517 ] DEBUG : Cluster start called. is_up=False (cluster.py:2797, start) 2026-09-30 15:22:18 [ 517 ] DEBUG : Docker networks for project roottestcheckings3blobsparanoid_gw2 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:22:18 [ 517 ] DEBUG : Docker containers for project roottestcheckings3blobsparanoid_gw2 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:22:18 [ 517 ] DEBUG : Docker volumes for project roottestcheckings3blobsparanoid_gw2 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:22:18 [ 517 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-09-30 15:22:18 [ 517 ] DEBUG : Docker networks for project roottestcheckings3blobsparanoid_gw2 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:22:18 [ 517 ] DEBUG : Docker containers for project roottestcheckings3blobsparanoid_gw2 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:22:18 [ 517 ] DEBUG : Docker volumes for project roottestcheckings3blobsparanoid_gw2 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:22:18 [ 517 ] DEBUG : Command:docker container list --all --filter name='^/roottestcheckings3blobsparanoid_gw2_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-09-30 15:22:18 [ 517 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-09-30 15:22:18 [ 517 ] DEBUG : No running containers for project: roottestcheckings3blobsparanoid_gw2 (cluster.py:904, cleanup) 2026-09-30 15:22:18 [ 517 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-09-30 15:22:18 [ 517 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-09-30 15:22:18 [ 517 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:22:18 [ 517 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-09-30 15:22:18 [ 517 ] DEBUG : Setup directory for instance: node (cluster.py:2810, start) 2026-09-30 15:22:18 [ 517 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/configs/storage_conf.xml'] to /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/configs/config.d (cluster.py:4636, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/database (cluster.py:4653, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/logs (cluster.py:4664, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log"] (cluster.py:4750, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Setup directory for instance: node_with_inf_s3_retries (cluster.py:2810, start) 2026-09-30 15:22:18 [ 517 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/configs/storage_conf.xml'] to /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/configs/config.d (cluster.py:4636, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/database (cluster.py:4653, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/logs (cluster.py:4664, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log"] (cluster.py:4750, create_dir) 2026-09-30 15:22:18 [ 517 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'MINIO_CERTS_DIR': '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/minio/certs', 'MINIO_DATA_DIR': '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/minio/data', 'MINIO_PORT': '9001', 'SSL_CERT_FILE': '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/minio/certs/public.crt'} stored in /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env (cluster.py:86, _create_env_file) 2026-09-30 15:22:18 [ 517 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-09-30 15:22:18 [ 517 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-09-30 15:22:18 [ 517 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-09-30 15:22:18 [ 517 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-09-30 15:22:18 [ 517 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 825 (connectionpool.py:547, _make_request) 2026-09-30 15:22:18 [ 517 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling node ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... done (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:23:53 [ 517 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:23:53 [ 517 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... Pulling proxy2 ... Pulling resolver ... Pulling minio1 ... Pulling node ... Pulling node_with_inf_s3_retries ... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling node ... pulling from altinityinfra/integr... Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... Pulling resolver ... pulling from altinityinfra/python... Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... Pulling node_with_inf_s3_retries ... status: image is up to date for a... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling node ... done Pulling node_with_inf_s3_retries ... done Pulling resolver ... done Pulling minio1 ... error Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy2 ... done ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:23:54 [ 517 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling node ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... done (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:25:28 [ 517 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:25:28 [ 517 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... Pulling proxy2 ... Pulling minio1 ... Pulling node_with_inf_s3_retries ... Pulling node ... Pulling resolver ... Pulling node ... pulling from altinityinfra/integr... Pulling resolver ... pulling from altinityinfra/python... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node ... done Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling proxy2 ... done Pulling resolver ... done Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... Pulling node_with_inf_s3_retries ... status: image is up to date for a... Pulling node_with_inf_s3_retries ... done Pulling minio1 ... error Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:25:31 [ 517 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... done (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling node ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:27:05 [ 517 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:27:05 [ 517 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... Pulling proxy2 ... Pulling minio1 ... Pulling node ... Pulling node_with_inf_s3_retries ... Pulling resolver ... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... Pulling node_with_inf_s3_retries ... status: image is up to date for a... Pulling proxy2 ... done Pulling node_with_inf_s3_retries ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling resolver ... pulling from altinityinfra/python... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling minio1 ... error Pulling node ... pulling from altinityinfra/integr... Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node ... done ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:27:08 [ 517 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... done (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling node ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:28:40 [ 517 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:28:40 [ 517 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... Pulling proxy1 ... Pulling resolver ... Pulling minio1 ... Pulling node ... Pulling node_with_inf_s3_retries ... Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... Pulling node_with_inf_s3_retries ... status: image is up to date for a... Pulling node_with_inf_s3_retries ... done Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy2 ... done Pulling resolver ... pulling from altinityinfra/python... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling resolver ... done Pulling proxy1 ... done Pulling minio1 ... error Pulling node ... pulling from altinityinfra/integr... Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node ... done ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:28:44 [ 517 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling node ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... done (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:30:16 [ 517 ] DEBUG : Failed to start cluster: (cluster.py:3178, start) 2026-09-30 15:30:16 [ 517 ] DEBUG : Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... Pulling proxy1 ... Pulling minio1 ... Pulling node ... Pulling node_with_inf_s3_retries ... Pulling resolver ... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling node ... pulling from altinityinfra/integr... Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy2 ... done Pulling proxy1 ... done Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node ... done Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... Pulling node_with_inf_s3_retries ... status: image is up to date for a... Pulling node_with_inf_s3_retries ... done Pulling minio1 ... error Pulling resolver ... pulling from altinityinfra/python... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:3179, start) 2026-09-30 15:30:16 [ 517 ] DEBUG : None (cluster.py:3180, start) 2026-09-30 15:30:16 [ 517 ] WARNING : docker-compose up was not called. Trying to export docker.log for running containers (cluster.py:3246, shutdown) 2026-09-30 15:30:16 [ 517 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-09-30 15:30:17 [ 517 ] DEBUG : Docker networks for project roottestcheckings3blobsparanoid_gw2 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:30:17 [ 517 ] DEBUG : Docker containers for project roottestcheckings3blobsparanoid_gw2 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:30:17 [ 517 ] DEBUG : Docker volumes for project roottestcheckings3blobsparanoid_gw2 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:30:17 [ 517 ] DEBUG : Command:docker container list --all --filter name='^/roottestcheckings3blobsparanoid_gw2_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-09-30 15:30:17 [ 517 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-09-30 15:30:17 [ 517 ] DEBUG : No running containers for project: roottestcheckings3blobsparanoid_gw2 (cluster.py:904, cleanup) 2026-09-30 15:30:17 [ 517 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-09-30 15:30:17 [ 517 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-09-30 15:30:17 [ 517 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:30:17 [ 517 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-09-30 15:30:17 [ 517 ] WARNING : docker-compose up was not called. Trying to export docker.log for running containers (cluster.py:3246, shutdown) 2026-09-30 15:30:17 [ 517 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-09-30 15:30:17 [ 517 ] DEBUG : Docker networks for project roottestcheckings3blobsparanoid_gw2 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:30:17 [ 517 ] DEBUG : Docker containers for project roottestcheckings3blobsparanoid_gw2 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:30:17 [ 517 ] DEBUG : Docker volumes for project roottestcheckings3blobsparanoid_gw2 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:30:17 [ 517 ] DEBUG : Command:docker container list --all --filter name='^/roottestcheckings3blobsparanoid_gw2_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-09-30 15:30:17 [ 517 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-09-30 15:30:17 [ 517 ] DEBUG : No running containers for project: roottestcheckings3blobsparanoid_gw2 (cluster.py:904, cleanup) 2026-09-30 15:30:17 [ 517 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-09-30 15:30:17 [ 517 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-09-30 15:30:17 [ 517 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:30:17 [ 517 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) ______ ERROR at setup of test_adaptive_timeouts[node_with_inf_s3_retries] ______ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ___________________ ERROR at setup of test_no_key_found_disk ___________________ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception __________ ERROR at setup of test_query_is_canceled_with_inf_retries ___________ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _______________ ERROR at setup of test_upload_after_check_works ________________ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ______ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[br] ______ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _____ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[bz2] ______ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ___ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[deflate] ____ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _____ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[gzip] _____ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _____ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[lz4] ______ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _____ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[none] _____ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ______ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[xz] ______ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _____ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[zstd] _____ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[br] _ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[bz2] _ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[deflate] _ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[gzip] _ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[lz4] _ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[none] _ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[xz] _ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[zstd] _ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _______ ERROR at setup of test_when_error_is_retried[connection_refused] _______ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _______ ERROR at setup of test_when_error_is_retried[qps_limit_exceeded] _______ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ___________ ERROR at setup of test_when_error_is_retried[slow_down] ____________ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ____ ERROR at setup of test_when_error_is_retried[total_qps_limit_exceeded] ____ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _______ ERROR at setup of test_when_s3_broken_pipe_at_upload_is_retried ________ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False] _ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True] _ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_when_s3_connection_reset_by_peer_at_upload_is_retried[False] _ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_when_s3_connection_reset_by_peer_at_upload_is_retried[True] _ [gw2] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling node ... E Pulling node_with_inf_s3_retries ... E Pulling resolver ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... done E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E Pulling minio1 ... error E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception __________ ERROR at setup of test_attach_table_from_s3_plain_readonly __________ [gw6] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: > cluster.start() test_attach_table_from_s3_plain_readonly/test.py:58: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/.env', '--project-name', 'roottestattachtablefroms3plainreadonly_gw6', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/.env', '--project-name', 'roottestattachtablefroms3plainreadonly_gw6', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node2/docker-compose.yml', 'pull'] return non-zero code 1: Pulling zoo1 ... E Pulling proxy2 ... E Pulling zoo3 ... E Pulling proxy1 ... E Pulling minio1 ... E Pulling resolver ... E Pulling zoo2 ... E Pulling node1 ... E Pulling node2 ... E Pulling node1 ... pulling from altinityinfra/integr... E Pulling zoo2 ... pulling from altinityinfra/integr... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling resolver ... pulling from altinityinfra/python... E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling node1 ... digest: sha256:2ea9da862c215fd684... E Pulling node1 ... status: image is up to date for a... E Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... E Pulling zoo2 ... status: image is up to date for a... E Pulling node1 ... done E Pulling zoo2 ... done E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling resolver ... done E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling zoo1 ... pulling from altinityinfra/integr... E Pulling node2 ... pulling from altinityinfra/integr... E Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... E Pulling zoo1 ... status: image is up to date for a... E Pulling zoo1 ... done E Pulling node2 ... digest: sha256:2ea9da862c215fd684... E Pulling node2 ... status: image is up to date for a... E Pulling node2 ... done E Pulling zoo3 ... pulling from altinityinfra/integr... E Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... E Pulling zoo3 ... status: image is up to date for a... E Pulling zoo3 ... done E Pulling minio1 ... error E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ---------------------------- Captured stderr setup ----------------------------- Traceback (most recent call last): File "/ClickHouse/tests/integration/helpers/cluster.py", line 2831, in start retry(log_function=logging_pulling_images)(run_and_check)( File "/ClickHouse/tests/integration/helpers/retry_decorator.py", line 28, in wrapper raise e File "/ClickHouse/tests/integration/helpers/retry_decorator.py", line 19, in wrapper func(*args, **kwargs) File "/ClickHouse/tests/integration/helpers/cluster.py", line 129, in run_and_check raise Exception( Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/.env', '--project-name', 'roottestattachtablefroms3plainreadonly_gw6', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node2/docker-compose.yml', 'pull'] return non-zero code 1: Pulling zoo1 ... Pulling proxy2 ... Pulling zoo3 ... Pulling proxy1 ... Pulling minio1 ... Pulling resolver ... Pulling zoo2 ... Pulling node1 ... Pulling node2 ... Pulling node1 ... pulling from altinityinfra/integr... Pulling zoo2 ... pulling from altinityinfra/integr... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling resolver ... pulling from altinityinfra/python... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling node1 ... digest: sha256:2ea9da862c215fd684... Pulling node1 ... status: image is up to date for a... Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... Pulling zoo2 ... status: image is up to date for a... Pulling node1 ... done Pulling zoo2 ... done Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling proxy2 ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling resolver ... done Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling zoo1 ... pulling from altinityinfra/integr... Pulling node2 ... pulling from altinityinfra/integr... Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... Pulling zoo1 ... status: image is up to date for a... Pulling zoo1 ... done Pulling node2 ... digest: sha256:2ea9da862c215fd684... Pulling node2 ... status: image is up to date for a... Pulling node2 ... done Pulling zoo3 ... pulling from altinityinfra/integr... Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... Pulling zoo3 ... status: image is up to date for a... Pulling zoo3 ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied ------------------------------ Captured log setup ------------------------------ 2026-09-30 15:23:20 [ 529 ] INFO : Running tests in /ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/test.py (cluster.py:2790, start) 2026-09-30 15:23:20 [ 529 ] DEBUG : Cluster start called. is_up=False (cluster.py:2797, start) 2026-09-30 15:23:20 [ 529 ] DEBUG : Docker networks for project roottestattachtablefroms3plainreadonly_gw6 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:23:20 [ 529 ] DEBUG : Docker containers for project roottestattachtablefroms3plainreadonly_gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:23:20 [ 529 ] DEBUG : Docker volumes for project roottestattachtablefroms3plainreadonly_gw6 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:23:20 [ 529 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-09-30 15:23:20 [ 529 ] DEBUG : Docker networks for project roottestattachtablefroms3plainreadonly_gw6 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:23:20 [ 529 ] DEBUG : Docker containers for project roottestattachtablefroms3plainreadonly_gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:23:20 [ 529 ] DEBUG : Docker volumes for project roottestattachtablefroms3plainreadonly_gw6 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:23:20 [ 529 ] DEBUG : Command:docker container list --all --filter name='^/roottestattachtablefroms3plainreadonly_gw6_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-09-30 15:23:20 [ 529 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-09-30 15:23:20 [ 529 ] DEBUG : No running containers for project: roottestattachtablefroms3plainreadonly_gw6 (cluster.py:904, cleanup) 2026-09-30 15:23:20 [ 529 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-09-30 15:23:20 [ 529 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-09-30 15:23:20 [ 529 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:23:20 [ 529 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-09-30 15:23:20 [ 529 ] DEBUG : Setup directory for instance: node1 (cluster.py:2810, start) 2026-09-30 15:23:20 [ 529 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/configs/config.xml'] to /ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node1/configs/config.d (cluster.py:4636, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node1/database (cluster.py:4653, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node1/logs (cluster.py:4664, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Setup directory for instance: node2 (cluster.py:2810, start) 2026-09-30 15:23:20 [ 529 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/configs/config.xml'] to /ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node2/configs/config.d (cluster.py:4636, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node2/database (cluster.py:4653, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node2/logs (cluster.py:4664, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-09-30 15:23:20 [ 529 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:b74d853ebe76', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/keeper3/coordination', 'MINIO_CERTS_DIR': '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/minio/certs', 'MINIO_DATA_DIR': '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/minio/data', 'MINIO_PORT': '9001', 'SSL_CERT_FILE': '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/minio/certs/public.crt'} stored in /ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/.env (cluster.py:86, _create_env_file) 2026-09-30 15:23:20 [ 529 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-09-30 15:23:20 [ 529 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-09-30 15:23:20 [ 529 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-09-30 15:23:20 [ 529 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-09-30 15:23:20 [ 529 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 825 (connectionpool.py:547, _make_request) 2026-09-30 15:23:20 [ 529 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/.env', '--project-name', 'roottestattachtablefroms3plainreadonly_gw6', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node2/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling node1 ... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling node2 ... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... done (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling node2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling node2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling node2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling node2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling node1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling node1 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling node1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling node1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:24:56 [ 529 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:24:56 [ 529 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/.env', '--project-name', 'roottestattachtablefroms3plainreadonly_gw6', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node2/docker-compose.yml', 'pull'] return non-zero code 1: Pulling zoo2 ... Pulling zoo1 ... Pulling proxy2 ... Pulling zoo3 ... Pulling proxy1 ... Pulling resolver ... Pulling minio1 ... Pulling node1 ... Pulling node2 ... Pulling zoo3 ... pulling from altinityinfra/integr... Pulling zoo1 ... pulling from altinityinfra/integr... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling zoo2 ... pulling from altinityinfra/integr... Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... Pulling zoo3 ... status: image is up to date for a... Pulling zoo3 ... done Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... Pulling zoo1 ... status: image is up to date for a... Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling zoo1 ... done Pulling proxy1 ... done Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... Pulling zoo2 ... status: image is up to date for a... Pulling proxy2 ... done Pulling zoo2 ... done Pulling node2 ... pulling from altinityinfra/integr... Pulling node2 ... digest: sha256:2ea9da862c215fd684... Pulling node2 ... status: image is up to date for a... Pulling node2 ... done Pulling node1 ... pulling from altinityinfra/integr... Pulling node1 ... digest: sha256:2ea9da862c215fd684... Pulling node1 ... status: image is up to date for a... Pulling node1 ... done Pulling resolver ... pulling from altinityinfra/python... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:24:57 [ 529 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/.env', '--project-name', 'roottestattachtablefroms3plainreadonly_gw6', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node2/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling node1 ... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling node2 ... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling node1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling node2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... done (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling node1 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling node1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling node2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling node2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling node1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling node2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:26:44 [ 529 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:26:44 [ 529 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/.env', '--project-name', 'roottestattachtablefroms3plainreadonly_gw6', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node2/docker-compose.yml', 'pull'] return non-zero code 1: Pulling zoo2 ... Pulling zoo3 ... Pulling zoo1 ... Pulling proxy1 ... Pulling proxy2 ... Pulling resolver ... Pulling minio1 ... Pulling node1 ... Pulling node2 ... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling zoo3 ... pulling from altinityinfra/integr... Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling node1 ... pulling from altinityinfra/integr... Pulling node2 ... pulling from altinityinfra/integr... Pulling proxy2 ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... Pulling zoo3 ... status: image is up to date for a... Pulling proxy1 ... done Pulling zoo3 ... done Pulling node1 ... digest: sha256:2ea9da862c215fd684... Pulling node1 ... status: image is up to date for a... Pulling node2 ... digest: sha256:2ea9da862c215fd684... Pulling node2 ... status: image is up to date for a... Pulling node1 ... done Pulling node2 ... done Pulling zoo1 ... pulling from altinityinfra/integr... Pulling resolver ... pulling from altinityinfra/python... Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... Pulling zoo1 ... status: image is up to date for a... Pulling zoo1 ... done Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling zoo2 ... pulling from altinityinfra/integr... Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... Pulling zoo2 ... status: image is up to date for a... Pulling zoo2 ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:26:46 [ 529 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/.env', '--project-name', 'roottestattachtablefroms3plainreadonly_gw6', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node2/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling node1 ... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling node2 ... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... done (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling node1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling node1 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling node1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling node1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling node2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling node2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling node2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling node2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:28:19 [ 529 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:28:19 [ 529 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/.env', '--project-name', 'roottestattachtablefroms3plainreadonly_gw6', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node2/docker-compose.yml', 'pull'] return non-zero code 1: Pulling zoo2 ... Pulling zoo1 ... Pulling proxy2 ... Pulling proxy1 ... Pulling minio1 ... Pulling resolver ... Pulling zoo3 ... Pulling node1 ... Pulling node2 ... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling zoo3 ... pulling from altinityinfra/integr... Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... Pulling zoo3 ... status: image is up to date for a... Pulling zoo3 ... done Pulling zoo1 ... pulling from altinityinfra/integr... Pulling zoo2 ... pulling from altinityinfra/integr... Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... Pulling zoo1 ... status: image is up to date for a... Pulling zoo1 ... done Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... Pulling zoo2 ... status: image is up to date for a... Pulling zoo2 ... done Pulling minio1 ... error Pulling resolver ... pulling from altinityinfra/python... Pulling node1 ... pulling from altinityinfra/integr... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling node1 ... digest: sha256:2ea9da862c215fd684... Pulling node1 ... status: image is up to date for a... Pulling node1 ... done Pulling node2 ... pulling from altinityinfra/integr... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling node2 ... digest: sha256:2ea9da862c215fd684... Pulling node2 ... status: image is up to date for a... Pulling node2 ... done Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy2 ... done ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:28:21 [ 529 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/.env', '--project-name', 'roottestattachtablefroms3plainreadonly_gw6', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node2/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling node2 ... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling node1 ... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling node1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... done (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling node1 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling node1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling node1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling node2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling node2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling node2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling node2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:30:06 [ 529 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:30:06 [ 529 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/.env', '--project-name', 'roottestattachtablefroms3plainreadonly_gw6', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node2/docker-compose.yml', 'pull'] return non-zero code 1: Pulling zoo3 ... Pulling proxy2 ... Pulling zoo2 ... Pulling proxy1 ... Pulling minio1 ... Pulling resolver ... Pulling zoo1 ... Pulling node2 ... Pulling node1 ... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling zoo3 ... pulling from altinityinfra/integr... Pulling node1 ... pulling from altinityinfra/integr... Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... Pulling zoo3 ... status: image is up to date for a... Pulling zoo3 ... done Pulling node1 ... digest: sha256:2ea9da862c215fd684... Pulling node1 ... status: image is up to date for a... Pulling zoo2 ... pulling from altinityinfra/integr... Pulling node1 ... done Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... Pulling zoo2 ... status: image is up to date for a... Pulling zoo2 ... done Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy2 ... done Pulling node2 ... pulling from altinityinfra/integr... Pulling zoo1 ... pulling from altinityinfra/integr... Pulling node2 ... digest: sha256:2ea9da862c215fd684... Pulling node2 ... status: image is up to date for a... Pulling node2 ... done Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... Pulling zoo1 ... status: image is up to date for a... Pulling zoo1 ... done Pulling resolver ... pulling from altinityinfra/python... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:30:11 [ 529 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/.env', '--project-name', 'roottestattachtablefroms3plainreadonly_gw6', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node2/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling node1 ... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling node2 ... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling node1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling node1 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling node1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling node1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling zoo2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling node2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling zoo1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling node2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling node2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling node2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling zoo3 ... done (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Failed to start cluster: (cluster.py:3178, start) 2026-09-30 15:32:05 [ 529 ] DEBUG : Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/.env', '--project-name', 'roottestattachtablefroms3plainreadonly_gw6', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node1/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_attach_table_from_s3_plain_readonly/_instances_0_gw6/node2/docker-compose.yml', 'pull'] return non-zero code 1: Pulling zoo1 ... Pulling proxy2 ... Pulling zoo3 ... Pulling proxy1 ... Pulling minio1 ... Pulling resolver ... Pulling zoo2 ... Pulling node1 ... Pulling node2 ... Pulling node1 ... pulling from altinityinfra/integr... Pulling zoo2 ... pulling from altinityinfra/integr... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling resolver ... pulling from altinityinfra/python... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling node1 ... digest: sha256:2ea9da862c215fd684... Pulling node1 ... status: image is up to date for a... Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... Pulling zoo2 ... status: image is up to date for a... Pulling node1 ... done Pulling zoo2 ... done Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling proxy2 ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling resolver ... done Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling zoo1 ... pulling from altinityinfra/integr... Pulling node2 ... pulling from altinityinfra/integr... Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... Pulling zoo1 ... status: image is up to date for a... Pulling zoo1 ... done Pulling node2 ... digest: sha256:2ea9da862c215fd684... Pulling node2 ... status: image is up to date for a... Pulling node2 ... done Pulling zoo3 ... pulling from altinityinfra/integr... Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... Pulling zoo3 ... status: image is up to date for a... Pulling zoo3 ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:3179, start) 2026-09-30 15:32:05 [ 529 ] DEBUG : None (cluster.py:3180, start) 2026-09-30 15:32:05 [ 529 ] WARNING : docker-compose up was not called. Trying to export docker.log for running containers (cluster.py:3246, shutdown) 2026-09-30 15:32:05 [ 529 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-09-30 15:32:05 [ 529 ] DEBUG : Docker networks for project roottestattachtablefroms3plainreadonly_gw6 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:32:05 [ 529 ] DEBUG : Docker containers for project roottestattachtablefroms3plainreadonly_gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:32:05 [ 529 ] DEBUG : Docker volumes for project roottestattachtablefroms3plainreadonly_gw6 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:32:05 [ 529 ] DEBUG : Command:docker container list --all --filter name='^/roottestattachtablefroms3plainreadonly_gw6_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-09-30 15:32:05 [ 529 ] DEBUG : No running containers for project: roottestattachtablefroms3plainreadonly_gw6 (cluster.py:904, cleanup) 2026-09-30 15:32:05 [ 529 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-09-30 15:32:05 [ 529 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-09-30 15:32:05 [ 529 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-09-30 15:32:05 [ 529 ] WARNING : docker-compose up was not called. Trying to export docker.log for running containers (cluster.py:3246, shutdown) 2026-09-30 15:32:05 [ 529 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-09-30 15:32:05 [ 529 ] DEBUG : Docker networks for project roottestattachtablefroms3plainreadonly_gw6 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:32:05 [ 529 ] DEBUG : Docker containers for project roottestattachtablefroms3plainreadonly_gw6 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:32:05 [ 529 ] DEBUG : Docker volumes for project roottestattachtablefroms3plainreadonly_gw6 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:32:05 [ 529 ] DEBUG : Command:docker container list --all --filter name='^/roottestattachtablefroms3plainreadonly_gw6_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-09-30 15:32:05 [ 529 ] DEBUG : No running containers for project: roottestattachtablefroms3plainreadonly_gw6 (cluster.py:904, cleanup) 2026-09-30 15:32:05 [ 529 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-09-30 15:32:05 [ 529 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-09-30 15:32:05 [ 529 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:32:05 [ 529 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) ============================== slowest durations =============================== 525.44s setup test_attach_table_from_s3_plain_readonly/test.py::test_attach_table_from_s3_plain_readonly 479.19s setup test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node] 298.06s setup test_attach_backup_from_s3_plain/test.py::test_attach_part[compact0] 139.81s setup test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible 102.69s setup test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False] 80.46s call test_cgroup_limit/test.py::test_cgroup_cpu_limit 76.14s setup test_backward_compatibility/test_normalized_count_comparison.py::test_select_aggregate_alias_column 61.84s setup test_allowed_client_hosts/test.py::test_allowed_host 59.96s setup test_broken_projections/test.py::test_broken_ignored 56.71s setup test_buffer_profile/test.py::test_buffer_profile 56.05s setup test_attach_table_normalizer/test.py::test_attach_substr 55.81s setup test_async_load_databases/test.py::test_dependent_tables 55.77s setup test_authentication/test.py::test_authentication_fail 55.46s setup test_block_structure_mismatch/test.py::test 54.75s setup test_config_hide_in_preprocessed/test.py::test_hide_in_preprocessed 52.64s setup test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts 51.90s setup test_covered_by_broken_exists/test.py::test_make_clone_covered_by_broken_detached_dir_exists 49.96s call test_config_xml_main/test.py::test_xml_main_conf 49.78s setup test_cleanup_after_start/test.py::test_old_dirs_cleanup 45.33s setup test_backward_compatibility/test_cte_distributed.py::test_cte_distributed 39.97s setup test_concurrent_queries_restriction_by_query_kind/test.py::test_insert 36.87s setup test_backup_restore_new/test_shutdown_wait_backup.py::test_shutdown_wait_backup 35.87s call test_backup_restore_new/test_shutdown_wait_backup.py::test_shutdown_wait_backup 31.00s call test_async_load_databases/test.py::test_dependent_tables 29.80s call test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts 25.10s call test_async_load_databases/test.py::test_dict_get_data 20.88s call test_concurrent_queries_restriction_by_query_kind/test.py::test_select 19.47s call test_broken_projections/test.py::test_broken_ignored 19.28s call test_broken_projections/test.py::test_mutation_with_broken_projection 17.06s call test_broken_projections/test.py::test_broken_on_start 16.88s call test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False] 15.18s call test_async_load_databases/test.py::test_multiple_tables 12.63s call test_broken_projections/test.py::test_disappeared_projection_on_start 11.23s call test_broken_projections/test.py::test_broken_projections_in_backups_2 10.38s call test_concurrent_queries_restriction_by_query_kind/test.py::test_insert 10.23s teardown test_backup_restore_on_cluster/test.py::test_tables_dependency 8.28s call test_broken_projections/test.py::test_broken_ignored_replicated 8.11s call test_broken_projections/test.py::test_broken_projections_in_backups_3 7.95s call test_backup_restore_on_cluster/test.py::test_tables_dependency 7.09s call test_cleanup_after_start/test.py::test_old_dirs_cleanup 6.94s teardown test_block_structure_mismatch/test.py::test 6.91s call test_backup_restore_on_cluster/test.py::test_replicated_database_async 6.73s teardown test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible 6.67s teardown test_covered_by_broken_exists/test.py::test_make_clone_covered_by_broken_detached_dir_exists 6.46s call test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True] 6.12s teardown test_concurrent_queries_restriction_by_query_kind/test.py::test_select 5.81s call test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts 5.67s call test_backup_restore_on_cluster/test.py::test_mutation 5.67s call test_backup_restore_on_cluster/test.py::test_required_privileges 5.54s teardown test_buffer_profile/test.py::test_default_profile 5.35s teardown test_allowed_client_hosts/test.py::test_denied_host 5.22s teardown test_backward_compatibility/test_normalized_count_comparison.py::test_select_aggregate_alias_column 5.20s call test_broken_projections/test.py::test_materialize_broken_projection 5.17s call test_covered_by_broken_exists/test.py::test_make_clone_covered_by_broken_detached_dir_exists 4.99s teardown test_backward_compatibility/test_cte_distributed.py::test_cte_distributed 4.88s call test_backup_restore_on_cluster/test.py::test_replicated_database 4.39s call test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def 4.39s teardown test_config_hide_in_preprocessed/test.py::test_hide_in_preprocessed 4.31s teardown test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts 4.26s call test_backup_restore_on_cluster/test.py::test_keeper_value_max_size 4.16s call test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert 4.00s call test_backup_restore_on_cluster/test.py::test_system_users 3.96s call test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True] 3.95s call test_backup_restore_on_cluster/test.py::test_replicated_table 3.91s call test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def 3.86s teardown test_authentication/test.py::test_authentication_pass 3.74s call test_backup_restore_on_cluster/test.py::test_system_functions 3.72s call test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database] 3.70s call test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath 3.70s teardown test_broken_projections/test.py::test_mutation_with_broken_projection 3.67s call test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica 3.62s call test_broken_projections/test.py::test_check_part_thread 3.62s call test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid] 3.61s call test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge 3.35s call test_attach_table_normalizer/test.py::test_attach_substr_restart 3.28s teardown test_cleanup_after_start/test.py::test_old_dirs_cleanup 3.23s call test_backup_restore_on_cluster/test.py::test_file_deduplication 3.23s call test_backup_restore_on_cluster/test.py::test_projection 3.21s call test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True] 3.07s call test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster 2.86s call test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False] 2.45s call test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes 2.42s teardown test_backup_restore_new/test_shutdown_wait_backup.py::test_shutdown_wait_backup 2.40s call test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty 2.26s call test_backup_restore_on_cluster/test.py::test_empty_replicated_table 2.25s teardown test_async_load_databases/test.py::test_multiple_tables 2.15s teardown test_attach_table_normalizer/test.py::test_attach_substr_restart 2.02s call test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster 1.83s teardown test_backup_restore_on_cluster/test.py::test_replicated_table 1.78s teardown test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts 1.78s teardown test_backup_restore_on_cluster/test.py::test_replicated_database_async 1.73s teardown test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database] 1.68s teardown test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath 1.68s teardown test_backup_restore_on_cluster/test.py::test_system_users 1.67s call test_backward_compatibility/test_normalized_count_comparison.py::test_select_aggregate_alias_column 1.63s teardown test_backup_restore_on_cluster/test.py::test_required_privileges 1.63s teardown test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert 1.63s teardown test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def 1.60s teardown test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge 1.58s teardown test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica 1.58s teardown test_backup_restore_on_cluster/test.py::test_replicated_database 1.58s teardown test_backup_restore_on_cluster/test.py::test_system_functions 1.58s teardown test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty 1.58s teardown test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid] 1.54s teardown test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster 1.54s call test_allowed_client_hosts/test.py::test_allowed_host 1.53s teardown test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def 1.53s teardown test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True] 1.53s teardown test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False] 1.53s teardown test_backup_restore_on_cluster/test.py::test_mutation 1.53s teardown test_backup_restore_on_cluster/test.py::test_empty_replicated_table 1.53s teardown test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True] 1.51s teardown test_backup_restore_on_cluster/test.py::test_keeper_value_max_size 1.49s teardown test_backup_restore_on_cluster/test.py::test_file_deduplication 1.49s teardown test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False] 1.48s teardown test_backup_restore_on_cluster/test.py::test_get_error_from_other_host 1.48s teardown test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True] 1.48s teardown test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster 1.48s teardown test_backup_restore_on_cluster/test.py::test_projection 1.43s teardown test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes 1.07s call test_config_hide_in_preprocessed/test.py::test_hide_in_preprocessed 0.90s call test_backup_restore_on_cluster/test.py::test_get_error_from_other_host 0.88s call test_authentication/test.py::test_authentication_pass 0.75s call test_block_structure_mismatch/test.py::test 0.74s call test_allowed_client_hosts/test.py::test_denied_host 0.74s call test_attach_table_normalizer/test.py::test_attach_substr 0.69s call test_backward_compatibility/test_cte_distributed.py::test_cte_distributed 0.54s call test_buffer_profile/test.py::test_default_profile 0.53s call test_buffer_profile/test.py::test_buffer_profile 0.43s call test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible 0.43s call test_authentication/test.py::test_authentication_fail 0.43s call test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible 0.00s setup test_cgroup_limit/test.py::test_cgroup_cpu_limit 0.00s setup test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True] 0.00s setup test_async_load_databases/test.py::test_dict_get_data 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True] 0.00s setup test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True] 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database] 0.00s setup test_backup_restore_on_cluster/test.py::test_empty_replicated_table 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries] 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert 0.00s setup test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty 0.00s setup test_backup_restore_on_cluster/test.py::test_get_error_from_other_host 0.00s setup test_backup_restore_on_cluster/test.py::test_system_functions 0.00s setup test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False] 0.00s setup test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_table 0.00s setup test_concurrent_queries_restriction_by_query_kind/test.py::test_select 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_database_async 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid] 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts 0.00s setup test_broken_projections/test.py::test_disappeared_projection_on_start 0.00s setup test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def 0.00s setup test_broken_projections/test.py::test_broken_ignored_replicated 0.00s setup test_broken_projections/test.py::test_broken_projections_in_backups_2 0.00s setup test_backup_restore_on_cluster/test.py::test_tables_dependency 0.00s setup test_backup_restore_on_cluster/test.py::test_system_users 0.00s teardown test_allowed_client_hosts/test.py::test_allowed_host 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def 0.00s setup test_authentication/test.py::test_authentication_pass 0.00s teardown test_broken_projections/test.py::test_broken_on_start 0.00s teardown test_concurrent_queries_restriction_by_query_kind/test.py::test_insert 0.00s teardown test_async_load_databases/test.py::test_dependent_tables 0.00s teardown test_attach_backup_from_s3_plain/test.py::test_attach_part[wide1] 0.00s setup test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes 0.00s setup test_allowed_client_hosts/test.py::test_denied_host 0.00s setup test_backup_restore_on_cluster/test.py::test_projection 0.00s teardown test_broken_projections/test.py::test_broken_ignored 0.00s setup test_config_xml_main/test.py::test_xml_main_conf 0.00s setup test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible 0.00s teardown test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible 0.00s setup test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True] 0.00s setup test_backup_restore_on_cluster/test.py::test_file_deduplication 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge 0.00s teardown test_config_xml_main/test.py::test_xml_main_conf 0.00s teardown test_attach_table_from_s3_plain_readonly/test.py::test_attach_table_from_s3_plain_readonly 0.00s setup test_backup_restore_on_cluster/test.py::test_keeper_value_max_size 0.00s setup test_broken_projections/test.py::test_check_part_thread 0.00s setup test_backup_restore_on_cluster/test.py::test_mutation 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node] 0.00s teardown test_attach_backup_from_s3_plain/test.py::test_attach_part[compact0] 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused] 0.00s setup test_backup_restore_on_cluster/test.py::test_required_privileges 0.00s teardown test_broken_projections/test.py::test_broken_projections_in_backups_2 0.00s setup test_broken_projections/test.py::test_broken_projections_in_backups_3 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works 0.00s setup test_attach_table_normalizer/test.py::test_attach_substr_restart 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_database 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster 0.00s teardown test_broken_projections/test.py::test_broken_projections_in_backups_3 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded] 0.00s teardown test_attach_backup_from_s3_plain/test.py::test_attach_part[compact1] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd] 0.00s teardown test_buffer_profile/test.py::test_buffer_profile 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip] 0.00s setup test_broken_projections/test.py::test_broken_on_start 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4] 0.00s teardown test_attach_table_normalizer/test.py::test_attach_substr 0.00s setup test_buffer_profile/test.py::test_default_profile 0.00s setup test_attach_backup_from_s3_plain/test.py::test_attach_part[compact1] 0.00s teardown test_attach_backup_from_s3_plain/test.py::test_attach_part[wide0] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2] 0.00s teardown test_cgroup_limit/test.py::test_cgroup_cpu_limit 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2] 0.00s setup test_attach_backup_from_s3_plain/test.py::test_attach_part[wide1] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True] 0.00s setup test_attach_backup_from_s3_plain/test.py::test_attach_part[wide0] 0.00s setup test_broken_projections/test.py::test_mutation_with_broken_projection 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br] 0.00s teardown test_broken_projections/test.py::test_disappeared_projection_on_start 0.00s teardown test_authentication/test.py::test_authentication_fail 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries 0.00s setup test_broken_projections/test.py::test_materialize_broken_projection 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded] 0.00s setup test_async_load_databases/test.py::test_multiple_tables 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz] 0.00s teardown test_broken_projections/test.py::test_materialize_broken_projection 0.00s teardown test_broken_projections/test.py::test_check_part_thread 0.00s teardown test_broken_projections/test.py::test_broken_ignored_replicated 0.00s teardown test_async_load_databases/test.py::test_dict_get_data =========================== short test summary info ============================ ERROR test_attach_backup_from_s3_plain/test.py::test_attach_part[compact0] - Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... ERROR test_attach_backup_from_s3_plain/test.py::test_attach_part[compact1] - Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... ERROR test_attach_backup_from_s3_plain/test.py::test_attach_part[wide0] - Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... ERROR test_attach_backup_from_s3_plain/test.py::test_attach_part[wide1] - Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/.env', '--project-name', 'roottestattachbackupfroms3plain_gw0', '--file', '/ClickHouse/tests/integration/test_attach_backup_from_s3_plain/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... ERROR test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node] ERROR test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries] ERROR test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk - Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw2', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw2/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... ERROR test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True] ERROR test_attach_table_from_s3_plain_readonly/test.py::test_attach_table_from_s3_plain_readonly PASSED test_authentication/test.py::test_authentication_fail PASSED test_attach_table_normalizer/test.py::test_attach_substr PASSED test_authentication/test.py::test_authentication_pass PASSED test_buffer_profile/test.py::test_buffer_profile PASSED test_buffer_profile/test.py::test_default_profile PASSED test_attach_table_normalizer/test.py::test_attach_substr_restart PASSED test_allowed_client_hosts/test.py::test_allowed_host PASSED test_allowed_client_hosts/test.py::test_denied_host PASSED test_broken_projections/test.py::test_broken_ignored PASSED test_async_load_databases/test.py::test_dependent_tables PASSED test_broken_projections/test.py::test_broken_ignored_replicated PASSED test_broken_projections/test.py::test_broken_on_start PASSED test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False] PASSED test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True] PASSED test_async_load_databases/test.py::test_dict_get_data PASSED test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True] PASSED test_broken_projections/test.py::test_broken_projections_in_backups_2 PASSED test_concurrent_queries_restriction_by_query_kind/test.py::test_insert PASSED test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica PASSED test_broken_projections/test.py::test_broken_projections_in_backups_3 PASSED test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes PASSED test_async_load_databases/test.py::test_multiple_tables PASSED test_broken_projections/test.py::test_check_part_thread PASSED test_backup_restore_on_cluster/test.py::test_empty_replicated_table PASSED test_backup_restore_on_cluster/test.py::test_file_deduplication PASSED test_backup_restore_new/test_shutdown_wait_backup.py::test_shutdown_wait_backup PASSED test_backup_restore_on_cluster/test.py::test_get_error_from_other_host PASSED test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible PASSED test_broken_projections/test.py::test_disappeared_projection_on_start PASSED test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible PASSED test_concurrent_queries_restriction_by_query_kind/test.py::test_select PASSED test_backward_compatibility/test_normalized_count_comparison.py::test_select_aggregate_alias_column PASSED test_backup_restore_on_cluster/test.py::test_keeper_value_max_size PASSED test_broken_projections/test.py::test_materialize_broken_projection PASSED test_backup_restore_on_cluster/test.py::test_mutation PASSED test_backup_restore_on_cluster/test.py::test_projection PASSED test_backup_restore_on_cluster/test.py::test_replicated_database PASSED test_broken_projections/test.py::test_mutation_with_broken_projection PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_async PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database] PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table PASSED test_backward_compatibility/test_cte_distributed.py::test_cte_distributed PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster PASSED test_covered_by_broken_exists/test.py::test_make_clone_covered_by_broken_detached_dir_exists PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge PASSED test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath PASSED test_cgroup_limit/test.py::test_cgroup_cpu_limit PASSED test_backup_restore_on_cluster/test.py::test_required_privileges PASSED test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False] PASSED test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True] PASSED test_backup_restore_on_cluster/test.py::test_system_functions PASSED test_backup_restore_on_cluster/test.py::test_system_users PASSED test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def PASSED test_config_xml_main/test.py::test_xml_main_conf PASSED test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty PASSED test_cleanup_after_start/test.py::test_old_dirs_cleanup PASSED test_backup_restore_on_cluster/test.py::test_tables_dependency PASSED test_block_structure_mismatch/test.py::test PASSED test_config_hide_in_preprocessed/test.py::test_hide_in_preprocessed ================== 65 passed, 35 errors in 712.84s (0:11:52) =================== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 445, in subprocess.check_call(cmd, shell=True) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_5862m6 --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=b74d853ebe76 -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 test_allowed_client_hosts/test.py::test_allowed_host test_allowed_client_hosts/test.py::test_denied_host test_async_load_databases/test.py::test_dependent_tables test_async_load_databases/test.py::test_dict_get_data test_async_load_databases/test.py::test_multiple_tables 'test_attach_backup_from_s3_plain/test.py::test_attach_part[compact0]' 'test_attach_backup_from_s3_plain/test.py::test_attach_part[compact1]' 'test_attach_backup_from_s3_plain/test.py::test_attach_part[wide0]' 'test_attach_backup_from_s3_plain/test.py::test_attach_part[wide1]' test_attach_table_from_s3_plain_readonly/test.py::test_attach_table_from_s3_plain_readonly test_attach_table_normalizer/test.py::test_attach_substr test_attach_table_normalizer/test.py::test_attach_substr_restart test_authentication/test.py::test_authentication_fail test_authentication/test.py::test_authentication_pass test_backup_restore_new/test_shutdown_wait_backup.py::test_shutdown_wait_backup 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False]' 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True]' 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True]' test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes test_backup_restore_on_cluster/test.py::test_empty_replicated_table test_backup_restore_on_cluster/test.py::test_file_deduplication test_backup_restore_on_cluster/test.py::test_get_error_from_other_host test_backup_restore_on_cluster/test.py::test_keeper_value_max_size test_backup_restore_on_cluster/test.py::test_mutation test_backup_restore_on_cluster/test.py::test_projection test_backup_restore_on_cluster/test.py::test_replicated_database test_backup_restore_on_cluster/test.py::test_replicated_database_async test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts 'test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database]' 'test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid]' test_backup_restore_on_cluster/test.py::test_replicated_table test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath test_backup_restore_on_cluster/test.py::test_required_privileges 'test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False]' 'test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True]' test_backup_restore_on_cluster/test.py::test_system_functions test_backup_restore_on_cluster/test.py::test_system_users test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty test_backup_restore_on_cluster/test.py::test_tables_dependency test_backward_compatibility/test_cte_distributed.py::test_cte_distributed test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible test_backward_compatibility/test_normalized_count_comparison.py::test_select_aggregate_alias_column test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts test_block_structure_mismatch/test.py::test test_broken_projections/test.py::test_broken_ignored test_broken_projections/test.py::test_broken_ignored_replicated test_broken_projections/test.py::test_broken_on_start test_broken_projections/test.py::test_broken_projections_in_backups_2 test_broken_projections/test.py::test_broken_projections_in_backups_3 test_broken_projections/test.py::test_check_part_thread test_broken_projections/test.py::test_disappeared_projection_on_start test_broken_projections/test.py::test_materialize_broken_projection test_broken_projections/test.py::test_mutation_with_broken_projection test_buffer_profile/test.py::test_buffer_profile test_buffer_profile/test.py::test_default_profile test_cgroup_limit/test.py::test_cgroup_cpu_limit 'test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node]' 'test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries]' test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded]' test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True]' test_cleanup_after_start/test.py::test_old_dirs_cleanup test_concurrent_queries_restriction_by_query_kind/test.py::test_insert test_concurrent_queries_restriction_by_query_kind/test.py::test_select test_config_hide_in_preprocessed/test.py::test_hide_in_preprocessed test_config_xml_main/test.py::test_xml_main_conf test_covered_by_broken_exists/test.py::test_make_clone_covered_by_broken_detached_dir_exists -vvv" altinityinfra/integration-tests-runner:cf95077f4609 ' returned non-zero exit status 1.